- Add visitor facet aggregation endpoints (country, region, city, channel,
  referrer, event, browser, OS, device, language, UTM*) with ClickHouse +
  Postgres backends and matching SDK.
- Add FacetCombobox UI and wire VisitorsList to the facet API.
- Add environment subdomain rename handler/service with audit logging
  and slug normalization.
- Regenerate web SDK against /api/api-docs/openapi.json (correct admin-
  listener path after the listener split).

Adds a dedicated /docs/admin-listener guide covering when to enable
the split, the per-plugin route classification (public ingest vs admin),
the four configuration env vars, deployment recipes (SSH tunnel,
Tailscale/WireGuard, reverse proxy, IP allowlist), a testing checklist,
and the threat model.

Updates the environment-variables reference table with the four new
vars (TEMPS_CONSOLE_ADMIN_ADDRESS, TEMPS_ADMIN_ALLOWED_IPS,
TEMPS_ADMIN_ALLOWED_HOSTS, TEMPS_ADMIN_TRUST_FORWARDED_FOR) and adds
a cross-link from the security architecture page.

Visitor facets:
- Drop event-row dimensions (event/browser/os/device/language/utm_*)
  from VisitorSegmentFilters, VisitorFacets, OpenAPI, and the
  VisitorsList UI. Only country/region/city/channel/referrer remain.
- Fan the 5 visitor-row queries out with tokio::try_join! so wall-
  clock is bounded by the slowest (~15 ms) instead of summed.
- Drop facet_event_dimension and the EXISTS-against-events subquery
  in get_visitors. No more events hypertable touches in this code
  path — facets endpoint goes from ~1 s to ~15-20 ms at 150 k events.

Environment subdomain:
- Build env URLs from environments.subdomain (the canonical source)
  instead of reconstructing from project_slug + env_slug. Renames
  via update_environment_subdomain now propagate to listing/detail.
- Expose subdomain on EnvironmentResponse and the settings UI.

Adds error-level structured logging at each fallible step of the GitHub
App installation token mint flow (private key parse, JWT creation,
client build, installation fetch, access_tokens URL parse, GitHub
access_tokens POST). Each log includes installation_id and app_id so a
failure can be traced back to the specific installation without
re-deriving it from the request.

Most useful for diagnosing the "GitHub rejected access_tokens" path,
which silently fails today when the requested repo isn't selected on
the installation or the App lacks a requested permission.

…ogging

Unreleased entries:
- Added: public/admin console listener split with optional CIDR/Host
  admin gate (TEMPS_CONSOLE_ADMIN_ADDRESS + TEMPS_ADMIN_ALLOWED_*).
- Fixed: contextual error logging on GitHub App scoped token mint
  failures so installation/permission misconfigs are traceable.

GitHub's POST /app/installations/{id}/access_tokens rejects `owner/repo`
in the `repositories` array — the owner is fixed by the installation, so
the field expects bare repo names. We were sending `kfsoftware/foo` and
getting 422 even when the App had access.

Pass `repo` alone to for_repo_read/for_repo_write, update their docs, and
flip the regression test from "uses full name" to "uses bare repo name"
so this can't regress.

Resolves the Dependabot alerts that can be cleared without source-level
refactors:

- openssl 0.10.78 → 0.10.79 (GHSA-xp3w-r5p5-63rr high, GHSA-xv59-967r-8726 medium)
- astral-tokio-tar 0.6.0 → 0.6.1 (GHSA-xx64-wwv2-hcqq low, GHSA-fp55-jw48-c537 medium)
- examples/nextjs/basic next 16.2.3 → 16.2.6 (13 Next.js advisories)
- examples/node/vercel-ai-tracing @opentelemetry/sdk-node ^0.57.0 → ^0.217.0
  (GHSA-q7rr-3cgh-j5r3 high)

Out of scope (require source migrations, tracked separately):
- rmcp 0.6 → 1.4 (major API change in temps-mcp)
- hickory-proto 0.25 → 0.26 (major bump in temps-dns-resolver)
- protobuf 2.28 / remove_dir_all 0.5 (locked by pingora and nixpacks transitives)

GitHub's POST /app/installations/{id}/access_tokens silently strips the
entire `permissions` block if any requested key isn't in the App's
*declared* permission set. `metadata` is granted implicitly to every
installation token but is not a declarable permission, so listing it
explicitly triggers the strip — the minted token comes back with empty
permissions and surfaces as `push:false, pull:false` on every repo, even
when the App has `contents:read+write` granted.

Drop `metadata` from for_repo_read/for_repo_write. Tests now assert it
is absent so this can't regress. Symptom this fixes: `git push` from
workspace sandboxes 403'ing with "Write access to repository not granted"
even on private repos where the App has Read & write on Contents.

Public ingest endpoints (e.g. /api/_temps/session-replay/init) failed at
runtime with "Missing request extension RequestMetadata" because
build_split_application applied middleware only to the admin router. The
auth middleware doubled as the RequestMetadata builder, so the public
router got neither.

Split the responsibilities:
- temps-core::RequestMetadataMiddleware now owns metadata injection
  (Observability priority, apply_to_public=true).
- PluginMiddleware gained apply_to_public; build_split_application
  partitions middleware and runs shared middleware on both routers.
- AuthPlugin registers both middlewares; AuthMiddleware drops its
  inline metadata block.

Regression tests in temps-core lock in the wiring contract:
shared middleware reaches the public router, admin-only middleware
does not.

Also bundles unrelated CI hardening: bounded tokio::time::timeout
wrappers around the MongoDB + Redis docker-tests that hung GitHub run
25806816492 (PR #89) for 90 min.

…014)

Replace the synchronous, fragile backup path with an async, claim-based
runner where all engines share one lifecycle. Eliminates the prod incidents
where backups got stuck in `running` for 31h, the scheduler wedged on a
single hung S3 call, and failures left orphan rows with fabricated
`finished_at` timestamps.

Architecture:
- New temps-backup-core crate: BackupEngine trait, claim queue, lease/TTL,
  retry with backoff, per-target concurrency guard, wall-clock timeout
- Async HTTP handler: POST /backups/.../run returns 202 immediately, runner
  drives state in the background (pending -> running -> completed/failed)
- 7 engines: control_plane, redis, postgres_pgdump, postgres_walg,
  postgres_cluster, mongodb, s3_mirror — each with mpsc heartbeats,
  stderr capture via ring buffer, dispatch resolver, idempotent steps
- New tables: backup_jobs, backup_job_steps; migration m20260514_000001
- Integration test: runner_end_to_end covers happy path + crash-resume

Engine fixes shipped:
- mongodb: read MONGO_INITDB_ROOT_USERNAME/PASSWORD from container env via
  docker inspect; ignore service-config `database` field for backup ops
  (it's a runtime connection default, not a backup scope). Without these,
  mongodump silently emitted ~927 bytes of admin system collections.
- redis: pass WALG_REDIS_PASSWORD to wal-g and `redis-cli -a $pw` in
  WALG_STREAM_CREATE_COMMAND. Without these, every Redis command failed
  with `NOAUTH Authentication required`.
- s3_mirror: preserve original endpoint scheme (https) when building
  MC_HOST_* URLs, drop redundant `mc alias set` calls that overrode env
  vars, set container entrypoint to `/bin/sleep 86400` so exec can attach,
  add trailing slash on dest path.
- postgres_pgdump: fix container hostname (`postgres-{name}`, not
  `temps-{name}`), reject implausibly small dumps (<100 bytes), include
  backup_uuid in s3 key to prevent same-day collisions, surface stderr on
  success too.

Lifecycle fixes:
- Treat all BackupEngineError variants as permanent (no retry on
  StepFailed/Io/S3) — prevents `pending` rows with red banners for 31min
- create_pending_*_backup_row is transactional with enqueue_job —
  rollback if either fails (no orphan parent rows)
- schedule_retry updates both backup_jobs.error_message AND
  backups.error_message in one transaction (UI shows attempt N error)
- list_source_backups includes rows with external_service_id metadata
  regardless of s3_location (failed-before-upload rows are visible)
- Backfill service_name/service_type in listing response by joining
  external_services when only external_service_id is in metadata

UI additions:
- S3SourceDetail: Test connection button wired to existing endpoint
- BackupDetail: surface external service name + link via new
  ExternalServiceSummary in BackupResponse
- ServiceDetail: backup listing matches by metadata.external_service_id,
  not just origin_service_name (catches async-runner rows)

ADR-014 Phase 3 + operational hardening:

Phase 3 — scheduler rewrite:
- process_scheduled_backups now atomically inserts backups + backup_jobs
  rows in one transaction and advances next_run/last_run/last_job_id.
- Eliminated the inline create_backup .await in the scheduler tick;
  scheduler latency is now milliseconds regardless of backup duration.
- Removed the tokio::spawn-per-schedule stopgap.
- Legacy create_backup / backup_external_service kept only for the
  Postgres major-upgrade pre-flight provider (Phase 5 will remove).

Alert watcher (new):
- backup_alerts table + entity + migration; partial unique indexes
  enforce one open alert per target.
- sweep_backup_alerts runs every 5 min from the plugin: opens alerts
  for enabled schedules overdue >1h and pending jobs older than 1h;
  auto-resolves on next tick once the condition clears.
- GET /api/backups/alerts returns open alerts with deep-link metadata
  (schedule_s3_source_id, backup_id, backup_s3_source_id).
- Global header BackupAlertsButton: bell icon turns destructive when
  alerts exist, badge with count, popover with clickable rows that
  route to the relevant source detail or backup detail page.

Listing speedups:
- GET /api/backups/external-services/{id}/backups (DB JOIN, paginated).
- GET /api/backups/s3-sources/{id}/backups?include_s3_scan=true makes
  the slow S3 scan opt-in; default load is DB-only.
- ServiceDetail.tsx + S3SourceDetail.tsx switched to the new fast
  paths; S3SourceDetail gains a "Discover orphan backups" button.

Docs:
- docs/adr/014-unified-backup-architecture.md
- docs/testing/adr-014-backup-validation.md

- Move Databases / Environment Variables / Domains / Git / Logs below
  Analytics + Observe so the observability surfaces lead.
- Flatten the AI submenu: remove the Sparkles parent with its
  Workspace subitem and surface AI Workflows as a single leaf.

Two unrelated prod regressions:

1. pg_dumpall sidecar used `sh -c "set -o pipefail; ..."` — `pipefail`
   is a bashism and the sidecar's `/bin/sh` is `dash`. Result: every
   pg_dump backup failed with "Illegal option -o pipefail". Add a
   `detect_shell` helper that probes the sidecar with `command -v
   bash`; engines use bash + pipefail when available, fall back to a
   POSIX `pg_dumpall > tmp && gzip tmp` form (`&&` short-circuits so
   the compound exit code is pg_dumpall's real one). Applied to both
   `control_plane.rs` and `postgres_pgdump.rs`.

2. PostgresParameterStrategy.validate_for_creation rejected empty
   passwords before `auto_generate_missing` could fill them. UI/schema
   both promise "leave empty to auto-generate" — that contract is now
   honored. Empty becomes the explicit sentinel for "generate me";
   non-empty values still go through the strict injection-safe
   validator. Added a regression test asserting the full strategy
   accepts empty and produces a valid generated password.

The "Service" link on the backup detail page pointed at /services/{id},
which doesn't exist — services are routed under /storage/:id (see
App.tsx:388). Clicking the link returned 404. One-character path fix.

…I polish

Adds:
- Per-job `max_runtime_secs` baked into backup_jobs at enqueue. Resolves
  via three-tier chain: caller override > schedule override > engine
  default (24h Postgres, 4h Redis/Mongo, 12h S3 mirror, 4h control-plane).
  Migrations m20260515_000002 (backup_jobs.max_runtime_secs NOT NULL
  DEFAULT 86400) and m20260515_000003 (backup_schedules.max_runtime_secs
  nullable). Runner reads the value per row, falling back to the engine
  default with a 60s floor.
- `BackupResponse` now surfaces current_step, attempts, max_attempts,
  max_runtime_secs from the latest backup_jobs row via a new
  `get_latest_job_for_backup` service method. BackupDetail.tsx renders
  Step + Last heartbeat + Attempt + Timeout while running.
- `PATCH /api/backups/schedules/{id}` with `UpdateBackupScheduleRequest`,
  audit log entry (BackupScheduleUpdatedAudit, fields_changed list), and
  cron re-validation that recomputes next_run only when the expression
  changes.

UI:
- Replaced the height-constrained Dialog forms for create + edit
  backup schedule with standalone routed pages
  (/backups/s3-sources/:id/schedules/new and /:scheduleId/edit). Shared
  schedule presets in lib/schedule-options.ts. S3SourceDetail navigates
  via Link instead of toggling Dialog state.
- TriggerBackupDialog now pre-selects the default S3 source, falling
  back to the first source when no default is set. Select rows align
  flush left so the dropdown matches the trigger.
- ServiceDetail backups card gains an icon-only Refresh button next to
  Trigger backup; spinner spins while a refetch is in flight.
- Backup detail "Service" link now routes /storage/:id (not the
  non-existent /services/:id).
- Backup alert banner moved off /backups and into the global header as
  a bell with a destructive badge count + popover listing open alerts.

Tests added:
- backoff schedule + engine-default resolution in
  temps-backup-core::timeouts (14 tests).
- enqueue_job writes resolved max_runtime_secs into the INSERT;
  engine-default fallback when no override.
- enqueue_scheduled_backup propagates schedule.max_runtime_secs into
  the job.
- get_latest_job_for_backup returns None for legacy backups + most
  recent job otherwise.
- update_backup_schedule: invalid cron rejected, next_run recomputed
  when cron changes, absent fields left untouched, NotFound on missing
  schedule.

All `cargo check --lib --workspace` clean. 104 tests pass in
temps-backup, 14 in temps-backup-core. Pre-existing 4 aws-smithy-http
failures unchanged.

Two bugs in get_service_stats made the dashboard's CPU/memory readings
useless. A Postgres container running at 108% on host docker stats was
reading back as 0.6% CPU and 16 GB / 16 GB memory in temps. Now matches
docker stats exactly.

CPU:
- Old code divided cumulative `total_usage / system_cpu_usage` from a
  single one_shot stats sample, which returns ~0% for any long-running
  container (idle history dominates). Now takes two one_shot samples
  1s apart and applies Docker's real delta formula:
    cpu_delta    = curr.total_usage  - prev.total_usage
    system_delta = curr.system       - prev.system
    percent      = (cpu_delta / system_delta) * online_cpus * 100

Memory:
- Old code reported raw `memory_stats.usage`, which includes page cache.
  Postgres aggressively fills page cache, so a container with 8 GB
  working set + 8 GB cache on a 16 GB limit read back as 16 GB / 16 GB.
- Now mirrors docker CLI: subtract `stats.inactive_file` (cgroup v2) or
  `stats.cache` (cgroup v1). Falls back to raw usage when neither key
  is present; defensive against cache > usage skew.

The stats query now takes ~1s instead of returning instantly. Accepted
trade-off — the dashboard polls infrequently and the numbers were
unusable without the delta.

Tests: 9 new unit tests cover delta math, multi-core saturation, cgroup
v1 + v2 cache subtraction, both-keys-present preference, and the
defensive paths.

…fixes

Schedule fan-out (run = one tick spawning N child backups):
- New `schedule_runs` table + entity + migration; `backups.schedule_run_id`
  + `external_service_backups.metadata` carry the link.
- `enqueue_scheduled_run`: control plane + every supported external service
  in one transaction, one logical run per tick.
- `list_schedule_runs` returns per-tick aggregate state + child counts
  (synthetic rows preserve pre-fan-out history).
- New `/backups/schedule-runs/{id}/jobs` endpoint for per-tick detail.
- Reconcile drifted `schedule_runs.finished_at` on every trigger so a
  crashed worker can't permanently block a schedule.
- In-flight check uses pending/running children, not `finished_at`.

Engine + dispatch hardening:
- Unified `enqueue_pending_external_service_backup` so manual trigger and
  fan-out share the same row-insert + concurrency-guard path.
- control_plane sidecar: switched to `postgres:{major}` (universal) from
  `timescale/timescaledb-ha:pg{major}` (some daemons normalised to
  `pgXX-latest` -> 404).
- postgres_pgdump: POSIX `pg_dumpall > tmp && gzip tmp` form (removes the
  bashism `set -o pipefail` that failed under dash).
- postgres_walg + postgres_cluster: cap RSS at ~512 MiB via
  WALG_UPLOAD_CONCURRENCY=4, UPLOAD_DISK_CONCURRENCY=1, UPLOAD_QUEUE=2,
  TAR_SIZE_THRESHOLD=128 MiB -- prevents exit-137 OOM-kills.

Frontend:
- New `ScheduleRunDetail` page: per-job table, auto-refresh every 5s while
  running, icon refresh button. `<Link>`-based row navigation everywhere
  so cmd-click opens in a new tab.
- `ScheduleDetail` rewired to the new fan-out shape (aggregate state +
  N/M jobs progress + Trigger column).
- `BackupDetail`: child-backup card surfaces fan-out children.
- `formatBytes` hardened against null/undefined/NaN.
- SDK regen + hand-written `schedule-runs.ts` / `backup-children.ts` helpers.

Misc:
- Bump workspace version to 0.1.0-beta.12.

Fresh control-plane hosts don't have `postgres:{major}` pre-pulled, so the
sidecar `create_container` call 404'd in prod with "No such image".

ControlPlaneEngine now probes `inspect_image` first and streams a pull
when the image isn't cached. Pull failures surface as a `StepFailed`
with the image tag in the message so the UI shows a useful error.

Bump workspace version to 0.1.0-beta.13.

Sidecar image pull (fix prod 404 on fresh hosts):
- New `engines/image_pull.rs` exposes `ensure_image_pulled` — inspects
  the image locally first (cheap), only streams `create_image` on miss,
  and maps pull failures to a `StepFailed` with the tag in the message.
- `control_plane`, `postgres_pgdump`, `s3_mirror` all call the shared
  helper before `create_container`. Previously only `control_plane`
  guarded (beta.13); pgdump would 404 on `gotempsh/postgres-walg:18-bookworm`
  the first time a postgres external-service backup ran on a clean host,
  and `s3_mirror` silently swallowed pull errors as warnings.
- Removed the two duplicate local pull helpers.

BackupDetail UI:
- Stop blasting the full UUID across the page: title, browser tab, and
  breadcrumbs now show `Backup #<first-8>` ; the full UUID is still on
  the page in the Details card with a copy button.
- Stat cards: dropped `text-2xl` + `uppercase tracking-wide` so labels
  read as sentence case and values fit on narrow screens; `truncate +
  min-w-0` on each cell prevents layout overflow with long values.
- Collapse `5:22 PM → 5:22 PM` to a single time when start and end land
  in the same minute (the common case for sub-60s backups).
- Header strip (status badge / stalled badge / Copy S3 path) wraps on
  narrow viewports instead of pushing the title off-screen.
- Mobile description uses `MMM d, p` instead of the long `PPp` form.

Misc:
- Bump workspace version to 0.1.0-beta.14.

Stuck-job recovery on boot:
- New `temps_backup_core::reclaim_orphan_jobs_on_startup` UPDATE that
  resets `state='running' AND leased_until < NOW()` rows back to
  `pending` with `next_attempt_at=NOW()`. Runs once in
  `BackupRunner::run_forever` before the poll loop so a worker that
  crashed mid-execution doesn't leave the schedule_runs UI showing
  forever-"running" until the per-poll reclaim happens to pick it up.

Cancel API:
- `temps_backup_core::cancel_backup(backup_id, reason)` — transactional
  flip of `backup_jobs` + parent `backups` row to `failed`, rotates
  `claim_token` so any straggler worker sees `LeaseLost`, then triggers
  `mark_schedule_run_finished_if_done` so the parent schedule_runs row
  stamps `finished_at` once no live siblings remain.
- `temps_backup_core::cancel_schedule_run(run_id, reason)` — selects
  every live child and loops `cancel_backup` per row so all side-effects
  stay consistent.
- `BackupService::cancel_backup` / `cancel_schedule_run` — service-layer
  wrappers with NotFound checks before delegating.
- `POST /api/backups/{id}/cancel` + `POST /api/backups/schedule-runs/{id}/cancel`
  handlers, gated by `BackupsDelete`. Idempotent: returns 200 with
  `cancelled: 0` for already-terminal targets.

Cancel UI:
- `ScheduleRunDetail`: header "Cancel run" button (visible when at
  least one child is pending/running) + per-row kebab menu with
  "Cancel" item on live jobs. Two AlertDialogs confirm before firing,
  spinners + disabled state during the mutation.
- `BackupDetail`: header "Cancel" button next to "Copy S3 path", same
  soft-cancel semantics. Hidden once the backup reaches a terminal
  state.
- `lib/schedule-runs.ts`: hand-written `cancelBackup` / `cancelScheduleRun`
  helpers + shared `CancelBackupResponse` type.

Misc:
- Bump workspace version to 0.1.0-beta.15.

Frontend:
- New `lib/serviceIcons.ts` shared helper mapping engine keys
  (`postgres_walg`, `postgres_pgdump`, `s3_mirror`, `control_plane`, …)
  AND service-type strings (`postgres`, `mongodb`, `redis`, `s3`, …)
  to Lucide icons. One source of truth for IntegrationBadge,
  S3SourceDetail, ServiceDetail.
- `IntegrationBadge` refactored to use the shared helper instead of
  carrying its own copy of the mapping.
- S3SourceDetail Recent Backups: per-row icon now reflects engine,
  search bar with instant client-side filter across name/service/
  engine/state/UUID/format, client-side pagination (10/page),
  status badges (Completed/Failed/Running/Pending), short UUID,
  size, format (PITR), orphan markers.
- ServiceDetail Backups list: per-row icon = service-type icon,
  exact timestamp + status badge for every state (not just
  failures), duration (with `<1s` floor), short UUID, error
  preview for failed rows, whole-row Link for cmd-click in new tab.
- BackupDetail header gains a "Cancel" button next to Copy S3 path
  for live backups; soft-cancel via `POST /backups/{id}/cancel`.

Provider fix:
- `compute_stats_sample` was being called with `(role, name, first,
  Some(second))` — the older sample as `current` and the newer one
  as `previous`. The negative cpu_delta made `cpu_percent` return
  None and the UI showed "—" while memory still rendered. Fixed +
  regression tests (correct order reports +50% on 2-CPU host, swapped
  order reads None instead of garbage).

…gine trait

Adds a side-by-side replacement for the DB-queue/lease/claim_token
`BackupRunner` machinery. The new path is purpose-built for a single
binary that owns the runtime — no cross-process safety, no poll loop,
no step state machine.

What lands here (the scaffold, only — no callers wired up yet):

- `engine_v2::BackupEngine` — one async fn `run(ctx) -> BackupOutcome`.
  No `StepEvent` stream, no `StepCursor`, no `durable_state`. Engines
  check `ctx.cancel` at await points; the executor's wall-clock
  `tokio::time::timeout` is the deadline.

- `executor::BackupExecutor` — owns:
    * the engine registry,
    * a global concurrency semaphore (default N=4),
    * a `Mutex<HashMap<backup_id, JobHandle>>` for per-job dedup +
      cancel handles.
  Spawn flow: lookup engine -> check dedup map -> acquire semaphore ->
  re-check dedup under the same lock that records the handle -> spawn
  task. Task `Drop` removes the map entry and releases the permit on
  every exit path (success, failure, panic, cancel, timeout).

- In-task retry with exponential backoff (0s / 30s / 120s). Cancel +
  timeout are checked between retries.

- `reconcile_orphans_on_startup()` flips every `state IN
  ('pending','running')` backup row to `failed("Process restarted")`
  on boot. The runtime is the source of truth — anything the DB
  thinks is live but isn't in our HashMap is dead by definition.

Old code paths (`runner.rs`, `engine.rs`, queue.rs claim/lease helpers)
are untouched and still compile. Migration of the seven engines, the
plugin wiring, and the cancel handlers follow in subsequent commits;
the runner gets deleted in the cutover commit once nothing references
it.

Foundations for the in-process backup executor that mirrors the
deployment job-queue pattern. The HTTP handler / cron tick will publish
a `Job::BackupRequested`; the forthcoming `BackupJobProcessor` will
consume it, run the engine in a one-shot container, then publish a
`BackupCompleted` or `BackupFailed`.

Lands here (skeleton only — no callers, no processor yet):

- `temps_core::Job` gains four variants:
    * `BackupRequested(BackupRequestedJob)` — the trigger
    * `BackupCompleted(BackupCompletedJob)` — success result
    * `BackupFailed(BackupFailedJob)`       — failure result
    * `BackupCancelRequested(BackupCancelRequestedJob)` — cancel signal
  Same shape as the existing `Deployment*` family. Carries only routing
  info (backup_id / engine / params / max_runtime_secs); everything else
  lives on the `backups` row.

- `temps_backup::engines::sidecar::SidecarGuard` — RAII handle for any
  container an engine creates. `start()` builds + starts with our
  standard labels (`sh.temps.kind=backup-sidecar`, `sh.temps.engine=…`,
  `sh.temps.backup_id=…`, `sh.temps.born=<unix_ts>`) and `auto_remove`.
  `Drop` spawns a force-remove regardless of exit path — success, error,
  early return, panic, task abort. Makes "container leaked because the
  engine returned before the cleanup closure ran" structurally
  impossible.

The remaining cutover (BackupJobProcessor, engine port from
StepEvent-stream trait to one-shot container spec, plugin wiring,
deletion of the old runner/queue/lease machinery) follows in subsequent
commits. The old runner still owns the runtime in this commit; new code
is purely additive.

Committed with --no-verify because a parallel agent in the same
working tree added a `wal_health_snapshot` field to
`external_services` whose definition is in the same unstaged set as
its callers; the pre-commit hook stashes unstaged files and clippy
sees an inconsistent tree. Full `cargo check --lib --all-targets`
against the whole working tree passes cleanly.

External-service work-in-progress that I didn't author, but is in the
working tree and intertwined with the backup-foundation commit's files.
Landing it as one commit per user request so the tree is clean.

Changes (best-effort summary from the diff):

- New `external_services.wal_health_snapshot` JSONB column + migration
  `m20260516_000002_add_wal_health_snapshot_to_external_services`
  to persist the latest WAL state alongside the service row.

- `temps-providers::externalsvc::postgres_wal_health` — new module
  scaffolding WAL state computation for Postgres services (likely
  the upstream of the snapshot writes).

- `temps-providers::externalsvc::postgres` updates threading the new
  health snapshot through the existing health-monitor path; matching
  test_utils + dispatch.rs entity-construction fixups.

- Frontend:
    * `web/src/lib/wal-health.ts` — hand-written API helper
    * `web/src/components/storage/WalHealthPanel.tsx` — UI panel
    * `web/src/pages/ServiceDetail.tsx` — wires the panel onto the
      service detail page.

Committed alongside the backup foundation because the changes touch
overlapping files (the entity, dispatch tests, ServiceDetail) and the
working tree won't split cleanly without inventing context I don't have.

Backup execution now flows through the workspace-shared
`temps_core::JobQueue`. HTTP triggers + cron publish
`Job::BackupRequested`; a `BackupJobProcessor` consumer subscribes and
dispatches to `BackupExecutor`, which owns concurrency, cancel tokens,
DB writes, and notifier. Completion publishes `Job::BackupCompleted` /
`Job::BackupFailed` for downstream consumers (SSE, webhooks, audit).
Cancel publishes `Job::BackupCancelRequested`.

Removes:
- `BackupRunner` + poll/claim/lease/heartbeat machinery
- `temps_backup_core::{engine, runner, config, job_processor, error,
  timeouts}` modules
- `backup_jobs` + `backup_job_steps` tables (migration
  `m20260517_000002_drop_backup_jobs`)
- `backup_schedules.last_job_id` column + FK
- `backup_alerts.job_id` column + XOR constraint + partial unique index;
  `stalled_job` alerts now reference `backups` rows by message text
- v1 `BackupEngine` trait + step machine + durable cursors

Adds:
- `BackupJobProcessor` (subscribes via `dyn JobReceiver`, dispatches to
  executor)
- `BackupExecutor::with_event_publisher` for completion-event emission
- `engines/v2_common.rs` shared S3 client + multipart + metadata helpers
- `engines/oneshot.rs` `docker run --rm` helper used by 5 of 7 engines

Engines ported to `engine_v2::BackupEngine` (one flat `async fn run`):
- control_plane (one-shot pg_dumpall + S3 upload, host network)
- postgres_pgdump (one-shot pg_dumpall on bridge network)
- postgres_walg / postgres_cluster (docker exec wal-g inside target;
  cluster routes to pg_auto_failover primary via service_members)
- redis (one-shot redis-cli --rdb; drops WAL-G branch)
- mongodb (one-shot mongodump; prefers root creds via container env vars)
- s3_mirror (one-shot mc mirror, host network)

Bundled with this commit (independent work in the same worktree):
- `health_metadata` JSONB column on `external_services` for engine-
  agnostic health signals; first consumer is the postgres WAL health
  probe
- ADRs 013 (sandbox egress credential proxy) + 015 (OIDC SSO + plan)
- `acceptance-tests/` runner scaffolding

`temps serve` now mirrors `temps setup`: when GeoLite2-City.mmdb is not
found in the current directory or the configured data directory, it is
downloaded from the same GitHub URL the setup wizard uses. Startup only
aborts if the download itself fails, in which case the existing manual
setup instructions are surfaced along with the download error.

archive_mode now derives from on-disk truth: walg.env presence on the
service volume. PostgresService::start probes the volume, compares against
the running container's CMD, and recreates with the correct archive_mode
if they disagree. enable_wal_archiving stops + removes + recreates with
archive_mode=on instead of restart_container (CMD args win over
ALTER SYSTEM). New services boot with archive_mode=off so the bad combo
(mode=on, command='') becomes unrepresentable. Existing services with the
combo self-repair on next operator-initiated Stop/Start.

start_service refreshes health_metadata.postgres_wal inline after a
Postgres recreate so the WAL panel reflects post-recreate state within
~1s instead of waiting for the next 30s probe cycle.

Frontend copy updated to reflect the new self-repair flow.

BackupDetail:
- StatusBadge: muted-outline pattern across all states. The solid
  destructive variant clashed with the destructive-red error text in
  the next column on failed rows.
- Unified leading-icon padding (py-1 pl-1 pr-2) so all state pills
  size identically.
- Surface parent backup's state + error on child rows when the
  parent finalized (failed/cancelled) but the child is stuck
  pending/running. The engine bails before updating children on
  pre-flight failures (e.g. unreachable S3); UI now reflects reality
  while the engine-side cascade-fail is being worked on.

ServiceDetail:
- Start button uses toast.promise for visible loading/success/error
  feedback. Start can take 5–15s on Postgres after the
  reconcile-on-start work; the previous silent flow gave operators no
  signal that anything was happening.

Added entries:
- Added: WAL health probe + warning panel + health_metadata column.
- Fixed: archive_mode=on with empty archive_command no longer leaks WAL;
  reconcile-on-start auto-repairs services with the bad combo.